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ANTIHISTAMINE  DRUGS  AND  PERFORMANCE  ON  TASKS 


OBJECTIVES:  (1)  To  develop  a  generalizable  performance  measurement  system  and  (2) 
to  evaluate  the  effects  of  selected  antihistamine  drugs  on  weapon  system  operators  performing 
tasks  related  to  Command,  Control  and  Communication  (C^)  missions  during  sustained  opera¬ 
tions. 


BACKGROUND:  The  Air  Force,  Army,  and  Navy  are  involved  in  a  joint  program  to 
determine  the  suitability  for  triservice  introduction  of  new  classes  of  drugs.  Part  of  these  evalua¬ 
tions  are  to  determine  the  impact  of  these  drugs  on  the  ability  of  humans  to  perform  complex 
C*  mission-related  tasks.  The  decision  was  made  to  conduct  a  portion  of  these  performance 
evaluations  studies  using  a  generic  task  crew  station  developed  for  use  during  Level  III  phase 
testing,  e.g.  simulation.  This  decision  was  based  on  the  ability  to  completely  control  the  testing 
environment  in  a  ground-based  facility  in  a  repeat^le  manner  and  to  use  a  properly  constructed 
workstation  and  network  capable  of  extracting  performance  data  that  is  time  correlated  with  the 
mission  scenario  events. 

Prior  to  this  triservice  program  the  Naval  Air  Test  Center  (NATC)  developed  a  generic 
simulation  system  architecture  that  was  uniquely  suited  to  support  the  generation  and  data  acqui¬ 
sition  requirements  of  the  drug  evaluation  program.  This  government-designed  system  was 
developed  around  a  unique  set  of  hardware  and  software  that  allows  multiple  processors  to  easily 
share  data  and  be  synchronously  dedicated  to  common  tasks.  The  Air  Force  adapted  the  Navy 
system  to  allow  mission  scenarios  to  be  performed  by  a  team  of  weapon  system  operators  that 
have  both  ground-based  and  airborne  applications  in  C’  mission-related  tasks. 

NATC  developed  a  unique  simulation  architecture  design  to  provide  a  system  of  generic  task 
work  stations  that  will  be  used  at  the  Armstrong  Laboratory  (USAFSAM)  to  evaluate  human  per¬ 
formance  on  a  variety  of  complex  decision-making  tasks.  NATC  procured  the  general  purpose 
system  hardware  to  produce  a  drug-screening  performance  evaluation  facility  based  on  two  pro¬ 
cessors.  One  of  these  processors  will  be  used  to  present  the  real  time  C?  tasks  to  the  subject. 
The  other  processor  will  be  used  to  interface  with  the  existing  physiological  instrumentation  sys¬ 
tems  at  the  Armstrong  Laboratory  to  monitor  the  status  of  the  subject  during  the  tests.  Existing 
NATC  technology  and  hardware  was  used  to  link  the  two  processors  and  provide  a  complete, 
user  frigidly  system. 

In  addition  to  providing  the  task  simulation  processing  and  data  collection  systems,  NATC 
also  provided  the  special  purpose  simulation  software,  interfaces,  and  control  systems  required 
by  Armstrong  Laboratory  to  conduct  the  performance  and  physiological  evaluations.  These 
included  a  network  of  four  generic  work  stations  that  will  be  used  to  present  tasks  to  the  subject, 
an  experimenter’s  control  console,  and  instrumentation  interfaces  to  collect  data  from  existing 
Armstrong  Laboratory  physiological  measurement  systems. 

APPROACH.  The  Command,  Control,  and  Communication  Generic  Workstations  (C*W) 
developed  with  Army  funding  during  FY  84-87  was  used  to  measure  the  combined  effects  of 
antihistamine  and  systematic  variations  in  cognitive  workload  (stress)  on  system  operator 


performance  during  sustained  operations.  The  C’GW  initial  configuration  designed  by 
Armstrong  Laboratory  specified  the  mission  scenarios,  tasks,  control  and  displays  that  would 
replicate  the  functions  of  the  Airborne  Warning  and  Control  Systems  (AWACS)  weapons 
director  (WD)  crew  stations.  The  AWACS  WD  team  function  was  chosen  as  the  complex  task 
because  it  contained  C’  task  elements  common  to  all  DOD  crew  mission  requirements.  Objec¬ 
tive  behavioral  measures  of  team  performance  were  developed  using  local  contractor  support  per¬ 
sonnel  (Systems  Research  Laboratory,  Inc.and  NTl,  Inc.)  and  training  instructors  from  Tinker 
AFB,  Oklahoma.  Finally,  research  protocols  were  submitted  for  human  use  committee  £q)proval 
along  with  a  subject  recruitment  plan. 

The  C^  missions  were  performed  by  rated  AWACs  military  personnel  under  either  the 
influence  of  Seldane,  Benadryl,  or  placebo  control  in  a  double-blind  study  to  determine  any  per¬ 
formance  decrements  and/or  physiological  debilitations  that  would  affect  successful  completion 
of  the  Air  Defense  mission.  The  data  was  analyzed  in  a  format  to  provide  operational  gui¬ 
dance  of  the  effects  of  antihistamines  on  complex  decision-making  tasks  under  sustained  opera¬ 
tions. 

The  last  C’  generic  workstation  system  was  delivered  to  USAFSAM  in  the  first  quarter  of 
FY  89  that  allowed  initiation  of  baseline  performance  evaluations  in  July  89.  These  initial  eval¬ 
uations  were  conducted  using  volunteer  subjects  from  the  S52nd  AW  AC  Training  Wing.  A  high 
priority  was  placed  on  the  use  of  AWACS  military  personnel.  Information  obtained  in  these  ini¬ 
tial  tests  was  used  to  further  define  the  hierarchical  levels  of  data  collection  and  unique  audio/ 
visual  capabilities  required  to  completely  evaluate  the  effects  of  antihistamine  drugs  on  team  per¬ 
formance  under  sustained  operations.  During  the  entire  period  NATC  and  Systems  Research 
Laboratory,  Inc.  continued  to  assist  USAFSAM  by  providing  updated  test  and  scenario  simula¬ 
tion  software  to  meet  changing  triservice  drug-«;reening  program  requirements  from  OMPAT 
for  similar  C^  missions  related  to  both  the  Army  and  Navy. 

ACCOMPLISHMENTS .  The  accomplishments  (results)  are  presented  as  an  annual 
summary  of  events  for  the  entire  reporting  period.  The  contractor  documentation  reports  in  the 
Appendices  give  more  detail  on  the  development  of  hardware,  software,  and  the  performance 
measurement  system.  The  scientific  results  were  reported  under  a  series  of  publications  under 
MIPR  90MM0503  due  to  early  termination  of  FY90  funding  for  MIPR  87MM7507. 


ANNUAL  SUMMARY  OF  WORK  ACCOMPLISHED 
1987  C^  Generic  Workstation  Delivery 

Funding  was  received  17  February  1987.  The  Aircrew  Evaluation  Sustained  Operations  Perfor¬ 
mance  (AESOP)  facility  was  formally  opened  in  April  1987.  A  major  contract  was  initiated  by 
the  Air  Force  to  provide  support  in  completing  production  of  the  C*  Generic  Workstations  to  be 
used  in  the  proposed  antihistamine  evaluation  of  C’  team  performance  on  a  variety  of  complex 
decision  making  tasks.  All  parts  for  the  construction  of  the  two  remaining  C*  Generic  Work¬ 
stations  (total  of  four)  were  ordered.  The  final  software  module  for  the  operationing  system  was 
specified.  System  software  documentation  responsibilities  between  the  Navy  Technical  Contract 
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Monitor  and  the  Air  Force  technical  support  contractor  was  finalized.  An  audio  communications 
distribution  system  and  speech  synthesizer  units  were  delivered  and  tested.  The  Generic 
Workstations  were  received  in  November  from  Systems  Research  Laboratories.  Complete 
system  documentation  for  all  hardware  components  and  operating  system  software  modules  were 
tested  against  the  specifications  and  approved. 

1988  Software  and  Scenario  Development  and  Evaluation 

A  meeting  was  held  at  Tinker  AFB  OK  to  coordinate  the  development  of  the  AW  AC  scenarios. 
The  director  of  operations  at  the  552nd  AW  ACS  Training  Wing  was  briefed  on  the  progress  and 
future  subject  requirements  for  AW  AC  crewmembers.  A  video  tape  was  made  of  the  scenario 
displays  to  be  us^  in  the  simulation  exercise  involving  the  weapon  system  directors.  Major 
advancements  in  interactive  target  software  modules  and  performance  measurement  methodology 
were  completed.  The  audio  distribution  system  and  the  final  C’  Generic  Workstation  was  deli¬ 
vered  in  June  1988.  A  system  test  of  all  scenario  software  by  the  contractor  was  completed 
during  August.  A  major  system  evaluation  of  the  C?  crewstations  was  accomplished  in 
September  using  AW  ACS  instructors  from  the  966th  Training  Squadron  and  weapon  director 
flight  personnel  from  the  964th  AWACS  Operational  Squadron  from  Tinker  AFB,  OK.  The  au¬ 
dio  communication  networks,  mission  scenarios,  pilot  simulators,  map  displays,  and  target 
models  were  successfully  evaluated  with  only  a  few  minor  software  changes  required.  The  sce¬ 
narios  and  the  associated  crewstation  hardware  were  verified  by  the  AWACS  crews  to  be  at  a 
level  of  fidelity  to  proceed  with  the  antihistamine  phase  of  testing  in  July  89. 

1989  Protocol  Approval  and  Antihistamine  Data  Collection  Completed 

Performance  measures  for  the  following  classes  of  tasks  were  specified;  mission,  system,  team, 
embedded,  and  individual.  A  generalizable,  hierarchical,  measurement  methodology  scheme  was 
devised  to  correlate  critical  scenario  events  with  communications,  switch  actions,  and  video  tape 
documentation  of  non-verbal  team  interactions  into  a  common  time-related  data  base.  The 
research  protocol  to  evaluate  the  effects  of  antihistamines  on  12  teams  of  weapon  system  control¬ 
lers  was  approved  by  Air  Force  Human  Use  Committee  and  submitted  to  USAMRDC,  SGRD/ 
HR,  Fort  Hetrick,  MD  for  final  review  in  April.  The  data  collection  phase  of  the  effects  of  ter- 
fenadine  (Seldane)  and  diphenhydramine  (Benadryl)  on  aircrew  performance  began  in  August 
and  was  completed  in  October  1989. 

1990  Task  Termination,  Transition,  and  Completion 

All  of  the  data  reduction,  analysis,  and  final  scientific  reports  were  completed  under  the  combi¬ 
nation  of  Air  Force  funding  and  90MM0503  due  to  early  termination  of  this  MIPR  87MM7507 
prior  to  receipt  of  scheduled  funding  for  FY90.  An  interim  briefing  and  a  preliminary  report 
was  submitted  at  the  Office  of  Military  Performance  Assessment  Technology  (OMPAT)  in 
Washington,  D.C.  in  May  1990.  Major  new  tasking  in  FY  90-91,  Air  Force  reorganization, 
and  Operation  Desert  Shield/Storm  delayed  this  Final  Report. 

CONCLUSION.  The  OMPAT  directly  benefitted  from  the  development  of  generalizable 
objective  team  performance  assessment  measures  in  complex  military  decision-making  tasks. 
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The  data  collected  under  the  antihistamine  protocol  provided  the  DOD  the  guidance  and  recom¬ 
mendations  for  the  most  effective  use  of  antihistamines  in  operational  settings. 


PUBLICATIONS  AND  DOCUMENTATION 
Government  Technical  Reports 

Eddy,  D.E.  (1989)  Selected  team  performance  measures  in  a  C’  environment:  an  annotated 
bibliography.  Technical  Report  USAFSAM-TR-87-25,  USAF  School  of  Aerospace  Medicine, 
Brooks  AFB,  Texas. 

Schiflett,  S.G.,  Strome,  D.R.,  Eddy,  D.R.,  &  Dalrymple,  M.A.  (1990)  Aircrew  evaluation 
sustained  operations  performance  (AESOP):  A  triservice  facility  for  technology  transition. 
USAFSAM-TP-90-26,  USAF  School  of  Aerospace  Medicine,  Brooks  AFB,  Texas. 

Contractor  Delivery  Order  Documentation  Reports 

Work  accomplished  by  the  Contractor  on  this  MIPR  was  organized  on  a  delivery  order  basis. 
A  core  group  of  on-site  personnel  were  assigned  to  the  delivery  orders  as  required.  Additional 
needs  were  met  through  off-site  SRL  capabilities,  subcontracts,  and  consulting  agreements.  A 
brief  description  of  each  applicable  delivery  order  follows: 

Delivery  Order  0001:  C^  Generic  Workstation  FacUity.  The  system  networks  and 
components  of  the  AESOP  facility  were  integrated,  operated,  documented,  and  maintained 
through  SRL  technical  support  and  materials.  This  included  operating  and  maintaining  the 
hardware,  software,  and  firmware  associated  with  the  C’GW  systems;  developing  software  to 
present  scenarios  on  the  workstations,  collect  data,  generate  graphics,  and  score  task 
performance;  performing  systems  analysis  and  configuration  planning  for  components  requiring 
integration  into  the  facility  networks;  conducting  human  factors  engineering  to  support  research 
in  individual  and  team  performance  under  sustained  operations  and  stressful  conditions. 

Delivery  Order  (MX12:  Performance  Testing  Under  Sustained  Operations.  SRL  performed 
a  study  to  acquire  data  on  cognitive  task  performance  of  human  subjects  working  under  sustained 
operations.  The  Unified  Tri-Service  Cognitive  Performance  Assessment  Battery  (UTC-PAB) 
was  acquired,  configured,  and  implemented  as  an  integral  part  of  Delivery  Order  0002. 
Research  included  developing  experimental  protocols;  acquiring  and  analyzing  data;  developing 
training  objectives;  conducting  experiments;  and  reporting  collected  data. 

Tasks  included  systems  analysis,  hardware  and  software  acquisition,  and  system  integration. 
Acquiring  and  qualifying  subjects  was  a  critical  element  in  this  delivery  order.  SRL  developed 
a  subject  pool  and  standard  procedures  for  subject  recruitment,  training,  medical  testing  and  han¬ 
dling.  The  subject  requirements  were  met  in  accordance  with  the  accepted  protocols  and 
complied  with  all  of  the  guidelines  of  AFR  169-3  in  reference  to  subjects. 
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Delivery  Order  0003:  Fabrication  of  C’  Generic  Workstation  (C?GW)  and  Scenario 
Development.  Two  additional  C^GWs  and  the  software  necessary  to  present  interactive 
scenarios  to  aircrew  members  were  constructed  and  installed  by  SRL.  Specific  tasks  included 
procuring  parts,  fabrication,  assembly,  and  testing  based  on  the  design  of  current  workstations. 
The  software  development  included  completing  and  documenting  existing  software  modules, 
creating  software  systems  to  allow  the  user  to  alter  the  scenario  presentation  interactively  in  real 
time,  and  completing  an  air  defense  scenario  to  present  on  the  workstations.  Delivery  order 
0003  also  required  developing  embedded  performance  measurement  methods  to  incorporate  into 
the  workstations  and  scenarios. 

Delivery  Order  0005:  Command,  Control  and  Communication  (C^  Generic  Workstation 
for  Airborne  Warning  and  Control  Systems  (AW ACS)  Weapons  Directors  (WDs).  SRL’s 
management,  professional,  and  technical  personnel  provided  the  materials,  services,  and  skills 
to  ensure  full  and  efficient  operation,  maintenance,  and  support  of  the  total  function  of  the 
C’GW  facility.  Installation  and  integration  of  C’GW  scenario  and  graphics  systems  and  their 
associated  peripheral  systems  (such  as  the  Audio  Distribution  System  Network)  was  done  in 
cooperation  with  Air  Force  personnel.  In  addition  to  the  hardware  and  software  requirements, 
integration  included  analyzing  needs,  determining  resources,  planning  for  maximum  use  through 
information  sharing  and  interfacing,  and  anticipating  growth  patterns.  SRL  enhanced  the 
capabilities  of  the  AESOP  facility  by  resolving  technical  shortfalls  in  the  existing  systems,  and 
standardizing  graphics  systems  and  software  to  those  in  Air  Force  and  Navy  laboratories 
performing  C’  work.  Scenario  development  occurred  within  the  restrictions  and  requirements 
of  research  protocols  initiated  under  this  delivery  order.  This  included  defining  the  scenario 
requirements;  developing  a  timed  sequence  of  events  and  a  voice  script;  integrating  the  events 
and  voice  script  into  a  scenario  file;  digitizing  those  portions  of  the  voice  script  to  be 
synthesized;  and  testing  the  scenario. 

SRL  provided  human  factors  engineering  support  to  write  a  research  protocol  to  assess  the 
effects  of  sustained  operations  and  antihistamine  medication  on  complex  decision-making 
performance  and  on  team  performance.  The  protocol  required  us  to  describe  subject  task 
variables;  to  specify  data  acquisition  and  data  analysis  procedures  for  individual  and  team 
performance  metrics;  and  to  develop  AWACS  scenarios,  scoring  algorithms,  and  embedded 
performance  tasks. 

Delivery  Order  0006:  Antihistamine  Drugs  and  Sustained  Performance.  SRL  conducted  a 
study  to  evaluate  the  sensitivity  and  reliability  of  selected  cognitive  and  psychomotor 
performance  tasks  to  the  effects  of  two  antihistamines  (Seldane  and  Benadryl)  taken  during 
sustained  operations.  These  measures  and  others  were  defined  and  employed  to  determine  the 
magnitude  of  each  drug’s  effect  on  Weapons  Director  individual  and  team  performance. 
Directing  this  study  comprised  providing  systems  analysis  and  integration;  recruiting  and 
overseeing  a  subject  pool;  selecting  and  implementing  software  and  hardware  systems  to  meet 
the  performance  task  requirements;  compiling  system  and  software  documentation;  conducting 
experiments  in  accordance  with  the  Statement  of  Work  (SOW);  and  reporting  the  methods  and 
results  of  research  protocols. 
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The  performance  network  was  expanded  by  attaching  a  communications  server  to  the  Nestar 
network  with  a  phone  line  and  Cc:Mail  for  communication  with  remote  performance  networks. 

File  transfer  between  the  Nestar  network  and  other  local  area  networks  was  accomplished  by 
connecting  the  Nestar  Hub  and  the  AESOP  Banyan  file  server.  Additional  performance  stations 
were  equipped  with  SRL  LabPak  cards  and  prototype  response  boxes  according  to  Air  Force 
requirements. 

SRL  acquired  additional  technical  expertise  in  AW  ACS  C^  mission  scenario  development.  Hie 
technical  expert  defined  the  scope  of  the  scenario,  wrote  the  time-based  event  and  voice  scripts, 
and  assisted  in  the  application  of  performance  measures  to  the  scenario  plan.  He  also  acted  as 
Senior  Director  in  the  experimental  sessions,  trained  simulation  pilots,  briefed  the  test  subjects, 
set  the  pace  for  the  simulations,  and  evaluated  the  performance  of  the  teams  using  his  own 
experience.  SRL  additionally  provided  experienced  simulation  pilots  to  aid  in  the  scenario 
presentation  during  the  duration  of  the  experiments. 


APPENDICES 

APPENDIX  A  -  C^  Generic  Workstation  and  Computer  Networks 
APPENDIX  B  -  Aircrew  Evaluation  Sustained  Operations  Facility 
APPENDIX  C  -  Research  and  Development  Software  Documentation 
APPENDIX  D  -  Performance  Measurement  Methodology 
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C’  GENERIC  WORKSTATIONS  AND  COMPUTER  NETWORKS 


INTRODUCTION 


Overview 

This  report  contains  the  general  description  of  hardware  and  software  systems  developed  under 
the  requirements  of  several  Delivery  Orders  from  FY87  through  FY89  on  Air  Force  contract 
F33615-87-D-0601. 

Due  to  the  large  number  of  systems  within  the  Crew  Performance  Function,  Aerospace  Research 
Branch,  Crew  Technology  Division,  and  School  of  Aerospace  Medicine  (USAFSAM),  it  is 
important  to  address  integration  on  a  regular  basis.  The  integration  requirements  change  as  the 
systems  mature,  as  technology  advances,  and  as  new  systems  either  supplement  or  replace 
existing  ones. 

This  report  presents  descriptions  of  those  major  systems  that  were  developed  with  a  combination 
of  OMPAT  funds  and  leveraged  Air  Force  funding  support.  Not  all  systems  were  developed 
during  this  period  of  work  but  were  utilized  or  improved  upon  to  accomplish  common  triservice 
goals.  Extensive  documentation  of  actual  system  operation  and  management  for  the  networks 
and  computer  systems  is  provided  primarily  by  the  vendors.  Copies  of  all  such  documentation 
are  located  in  the  Aircrew  Evaluation  Sustained  Operations  Performance  (AESOP)  facility. 
Information  in  this  report  consists  of  only  general  descriptions,  specifications  summaries,  and 
documentation  of  technology  evolution.  In  addition  to  systems  information,  this  report  includes 
the  specifications  for  the  AESOP  facility. 

Mi^or  Systems 

The  major  systems,  covered  in  this  report,  that  currently  require  integration  and  systems  analysis 
are  described  briefly  below: 

VAX  research  network:  The  AESOP  VAX  research  network  is  dedicated  to  the  pre¬ 
sentation  of  scenarios  to,  and  collection  of  data  from,  the  Command,  Control  and 
Communication  Generic  Workstations.  It  also  serves  the  requirements  for  statistical 
analysis,  analog-to-digital  (A/D)  and  digital-to-analog  (D/A)  conversions,  database 
management,  forms  management,  and  programming  functions. 

Banyan  office  network:  The  Banyan  (Banyan  Systems,  Inc,  Westborough,  MA) 
network  consists  of  four  file  servers,  strategically  located  throughout  the  Division, 
that  provide  software  for  office-related  functions  as  well  as  some  programming, 
statistical,  and  data  reduction  capability.  Resources  are  shared  across  the  network, 
along  with  electronic  mail  and  message  services. 

A/D  and  D/A  acquisition  and  analysis  systems:  This  service  is  provided  as  part  of 
the  VAX  network.  Physiological  and  electrophysiological  recordings,  as  well  as 
audio  inputs,  can  be  digitized,  analyzed,  and  reproduced  with  this  system. 


Communications  servers:  Information  generated  in  the  AESOP  facility  and 
Aerospace  Research  Branch  is  exchanged  with  remote  sites  across  the  nation  using 
communication  packages.  ProComm  is  used  for  simple  file  transfers.  cc:Mail  is 
an  integrated  software  system  designed  for  network-ba^  unattended  electronic  mail 
message  services  and  file  transfer. 

Electronic  mail  systems:  Every  network  has  its  own  electronic  mail  package.  These 
are  currently  restricted  to  the  host  networks  and  do  not  share  resources. 

Visual  and  audio  recording  systems:  The  simulations  presented  to  subjects  on  the 
C’  workstations  involve  both  verbal  and  non-verbal  communication.  The  Audio 
Distribution  System  (ADS),  designed  and  built  by  SRL  for  the  AESOP  application, 
allows  recording  of  individual  voices  during  the  scenario.  Low  light  level  video 
recordings  were  also  used  to  document  crew  behavior. 

Desktop  Publishing  systems:  These  systems  support  the  creation  of  presentation 
copies  of  technical  reports,  documentation,  and  briefing  materials  generated  by  the 
A^OP  facility  and  the  Branch. 

SYSTEM  DESCRIPTIONS 

The  major  systems,  that  required  integration  and  systems  analysis  during  the  reporting  period 
from  1987  through  1989  are  described  briefly  below: 

-  Command.  Control  and  Communication  Generic  Workstation  (Attachment  2,  Se¬ 
quence  1,  Delivery  Order  0003):  The  C’GW  is  a  multiple  task  simulation  system 
used  to  develop  measurement  methodologies  to  assess  team  performance  effective¬ 
ness  in  a  simulated  C^  environment.  The  overall  goal  is  to  improve  aircrew  safety 
and  mission  effectiveness  by  transition  of  team  performance  effectiveness  measures 
from  the  laboratory  to  field  test  environments.  The  technical  objective  is  to  generate 
a  mission  scenario  on  a  generic  crewstation  that  places  realistic  task  demands  on 
weapon  system  operators  in  a  simulated  C^  environment.  Initially  this  system  was 
used  to  provide  mission  scenarios,  tasks,  controls,  and  displays  replicating  the  func¬ 
tions  of  the  Airborne  Warning  and  Control  System  (AW ACS)  weapons  director 
(WD)  crewstation  to  evaluate  the  effects  of  antihistamines  on  crew  performance. 

Nestar  performance  network  (Attachment  2,  Sequence  1 ,  Delivery  Order  0002):  The 
Nestar  network  consists  of  two  file  servers,  a  communications  server,  a  print  server, 
and  five  personal  computers  (PCs)  in  a  star  configuration  operating  under  the  Nestar 
networking  system  using  Datapoint’s  ARCnet  protocol.  It  provides  performance  task 
presentation  and  data  collection. 

Audio  Distribution  System  (Attachment  2,  Sequence  1,  Delivery  Order  0003):  The 
Audio  Distribution  System  (ADS)  consists  of  the  following: 

three  intercom  networks; 
eight  simulated  radio  channels; 

one  guard  channel  for  communications  from  the  experimenter; 
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a  keyed  microphone  for  intermittent  transmission; 

two  speech  synthesis/recognition  devices  (Votan  VTR-6050)  that  will 

simulate  a  portion  of  the  AWACS  communications; 

10  identical  nodes,  each  of  which  can  be  configured  to  simulate  any 
functional  position. 

Stations  on  the  ADS  network  are  connected  in  a  star  network  to  a  central  control 
hub.  The  hub  is  the  experimenter’s  station  and  the  point  of  control  for  system 
communications.  The  experimenter  can  establish  communication  with  any  node 
while  remaining  transparent  to  the  simulation.  There  is  a  white  noise  generation 
node  that  allows  the  addition  of  noise  at  any  volume  on  any  communications 
channel.  Output  from  the  ADS  is  channeled  to  a  multi-channel  recorder  and 
recorded  both  by  individual  voice  and  by  selected  channels. 

Graphics  processors  and  workstations  (Attachment  2,  Sequence  1,  Delivery  Order 
0(X)3):  The  scenarios  and  performance  tasks  presented  to  subjects  require  a  signifi¬ 
cant  amount  of  graphics  development  and  display.  Both  personal  computer  and 
C^GW  graphics  engine  systems  were  upgraded  to  VGA  monitors  and  Silicon  Gra¬ 
phics  4D/50  workstations,  respectively.  This  was  upgrade  was  necessary  as  techno¬ 
logy  advanced  to  provide  increased  capability  for  accomplishing  the  research  as  out¬ 
lined  in  the  protocols.  However,  advanced  technology  was  not  chased  as  an  end  in 
itself;  rather  it  was  pursued  to  meet  the  immediate  basic  requirements  of  the  contract 
in  a  cost  effective  manner. 
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NETWORKS 


Background 

Prior  to  1986,  in-house  computing  resources  for  the  Crew  Performance  Function  consisted  of 
a  few  general-purpose  and  specialized  microcomputers  and  two  POP  1  l/34s  used  for  A/D  data 
acquisition,  D/A  processing,  and  performance  task  presentation.  Other  systems  were  centralized 
in  the  computer  rooms  of  the  School  of  Aerospace  Medicine.  The  centralized  resources 
provided  text  editing,  statistical  analysis,  and  programming  functions.  However,  they  were 
greatly  overburdened  and  poorly  managed. 

Several  significant  developments  occurred  impacting  the  resource  needs  of  the  Branch  and  the 
Division.  First,  the  initial  C*GW  systems  were  completed  and  brought  two  dedicated  VAX 
11 /780s  and  their  associated  peripherals  to  the  Branch.  Second,  the  Total  In-Flight  Simulator 
CTIFS)  program  that  evaluted  the  effects  of  pyridostigmine  bromide  on  inflight  performance  had 
concluded,  and  the  resulting  statistical  analysis  requirements  were  enormous.  The  available 
resources  were  inadequate,  and  plans  were  initiated  to  put  statistical  analysis  capability  under 
local  control.  The  TIFS  data  analysis  also  revealed  the  limitations  of  the  A/D  systems  available 
to  the  Branch.  Systems  were  old,  incompatible,  and  limited  in  size  and  sp^.  Air  Force 
planning  and  funding  was  begun  to  provide  these  services  within  the  Division. 

The  distributed  processing  concept  continued  to  grow  in  this  and  other  branches  and  divisions 
at  USAFSAM.  As  the  plans  developed,  a  contract  was  established  between  the  Air  Force  and 
Digital  Equipment  Corporation  (DEC),  providing  computers  at  very  low  prices.  As  a  result  of 
the  sudden  availability  of  MicroVAX  systems  in  multiple  configurations,  several  systems  were 
purchased  and  added  to  the  computing  facilities  of  the  Branch  and  Division.  These  systems  were 
assigned  functions  according  to  their  architecture  and  made  available  by  the  Air  Force  to  support 
OMPAT  projects.  They  were  configured  and  integrated  into  the  Research  Network. 

In  addition  to  the  increased  need  for  large-system  computing  resources,  the  Branch  was 
experiencing  the  trend  toward  PC-based  needs.  The  Function  had  one  PC/XT  compatible  Zenith 
150  and  a  few  Zenith  100  systems  used  for  word  processing,  spreadsheets,  and  low-level  data 
reduction  and  analysis.  Several  Apple  II  and  NEC  PCs  were  used  for  performance  battery 
presentation  both  in-house  and  at  remote  sites.  The  OMPAT  set  a  "benchmark"  by  targeting  the 
IBM  compatible  systems  for  performance  battery  upgrades  for  both  the  Level  I  physiological) 
and  Level  II  (performance)  batteries  that  included  the  Unified  Triservice  Cognitive  Performance 
Assessment  Battery  (UTC-PAB).  There  was  also  a  technological  upsurge  in  computing 
capabilities  versus  price.  With  an  increase  in  staff  and  the  implementation  of  another 
requirements  contract  with  Zenith  for  PC/AT  compatible  Zenith-248s,  the  stage  was  set  for  a 
PC  network.  The  Banyan  network  supplied  by  the  Air  Force  was  configured  for  office-based 
functions  and  minor  data  handling  and  programming.  The  Nestar  network  supplied  with 
OMPAT  funding  was  set  up  for  performance  research.  These  networks  have  extended  to 
Division-level  support  and  are  remaining  faithful  to  the  distributed  processing  concept  of 
sq)arating  word  processing  functions  from  performance  data  acquisition  and  analysis. 

The  next  section  presents  the  specific  network  upgrades  made  to  accommodate  these  expanding 
computer  resources. 
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VAX  Research  Network 


The  VAX  network  contains  a  tightly-coupled  VAXcluster  and  specialized  MicroVAX  systems 
connected  to  each  other  and  to  DEC  remote  clusters  and  systems  via  DECnet  Ethernet  and 
asynchronous  twisted  pair.  Figure  1  represents  the  systems  in  the  AESOP  and  local  area 
networks  in  and  around  the  Aircrew  Ev^uation  Sustained  Operations  Performance  (AESOP) 
facility.  One  of  the  requirements  established  by  OMPAT  in  preparation  for  obtaining  ^ta  from 
this  research  was  to  complete  the  integration  of  the  Performance  Network  to  support  file  transfer 
to  local  and  remote  systems  for  data  reduction  and  analysis. 

Systems  Performance  Network 

The  Performance  Network,  Figure  2,  provides  computerized  performance  test  battery 
presentation  and  data  collection  as  well  as  communication  with  other  performance  laboratories 
nationwide.  It  is  comprised  of  several  performance  workstations  (^nith  Z-248  microcom¬ 
puters),  two  file  servers  (Nestar  Plan  4000  and  Planstar),  a  communications  server  (PC-clone 
with  modem)  and  a  print  server  connected  to  a  16  channel  hub  in  a  star  configuration.  The  net¬ 
work  operates  using  the  ARCnet  protocol. 

The  Crew  Technology  Administrative  Network,  Figure  3,  provides  word  processing,  spread¬ 
sheets,  desktop  publishing,  electronic  mail,  time  management  and  data  bases  for  the  office  envi¬ 
ronment.  It  is  comprised  of  four  Banyan  file  servers  (a  Desk  Top  Server  [DTS]  and  three  286- 
based  servers)  serving  four  area  networks  of  Zenith  Z-248  and  Macintosh  systems.  The  file 
servers  contain  electronic  mail,  asynchronous  serial  ports,  shared  printing  resources  and  shared 
software,  including  Timeline,  WordPerfect  5.0,  Lotus  123  and  dBase  IV.  To  assist  in  the  prepa¬ 
ration  of  presentations  and  reports  this  network  also  has  access  to  an  Air  Force  Macintosh-ba^ 
publishing  workstation  with  color  printer  and  slide  generation  capability.  It  uses  thin- wire  coaxial 
connections  with  Interlan  or  3-Com  Ethernet  communications  boards. 

The  Cluster  and  Simulation  Network,  Figure  4,  provides  programming,  word  processing, 
simulation  generation  and  presentation,  data  base,  statistical,  and  realtime  analog/digital 
c£q)abilities  for  the  research  scientists.  The  VAX  cluster  consists  of  two  VAX  ll-780s  with 
simulation  software;  two  MicroVAX  Ills,  one  with  SAS  and  one  with  FOCUS;  and  a 
VAXstation  III/GPX  with  analog-to-digital  and  digital-to-analog  processing  and  high  level 
graphics  capability.  A  separate  MicroVAX  II  with  WPS  word  processing  will  be  added  to  the 
cluster  in  the  near  future.  There  are  both  shared  and  system-lo^  disks  and  printers.  Systems 
communicate  over  standard  Ethernet  coaxial  cable  using  the  DECnet  protocol. 

Other  distributed  VAX-based  networks  at  Brooks  Air  Force  Base  combine  with  these  to  form 
the  large  area  network,  including  resources  at 

-  Human  Systems  Division  (HSD), 

-  the  Occupational  and  Environmental  Health  Laboratories  (OEHL), 

-  other  USAFSAM  divisions 

-  the  Defense  Data  Network  (DDN)  tap. 

Network  Interconnections  and  File  Transfer 
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The  Banyan  network  is  connected  to  the  VAX  network  through  asynchronous  serial  ports  on  the 
Banyan  DTS  file  server.  The  eight  Banyan  ports  are  connected  by  two  eight-channel  asynchro¬ 
nous  ntultiplexers  to  DEC  terminal  server  ports.  A  proprietary  VTIOO  terminal  emulator  gives 
up  to  eight  microcomputers  on  the  Banyan  simultaneous  access  to  the  VAX  network  through 
these  lines.  Each  terminal  can  operate  at  up  to  9600  baud.  The  Kermit  protocol  is  used  for  file 
transfers  between  the  VAX  and  Banyan  systems.  As  a  result  of  this  integration  it  is  possible  to 
move  data  between  personal  computers  and  VAX-based  systems  with  more  powerful  resources. 

The  UTC-PAB  and  associated  performance  batteries  execute  on  7enith  Z-248  microcomputers 
connected  to  the  Performance  Network  Hub.  Each  workstation  has  both  hard  disk  and  5-1/4" 
floppy  diskette  drives.  Data  collected  during  the  tests  are  stored  in  files  on  the  local  system. 
The  long-range  plan  is  to  have  the  performance  batteries  and  data  storage  on  the  Performance 
Network  file  servers.  A  variety  of  statistical,  data  base  and  spreadsheet  functions  reside  on  the 
Administrative  Network  File  Server  and  on  the  MicroVAX  III  systems  in  the  AESOP  computer 
room.  Data  transfer  between  systems  can  be  accomplished  by  several  methods,  including: 
physical  transfer  of  a  floppy  diskette,  serial  RS232  connection  directly  to  one  other  local  host 
system  using  terminal  emulation  and/or  a  standard  transfer  protocol,  transfer  via  modems  and 
communication  software  packages  to  local  and  remote  systems,  and  connection  to  a  network  with 
appropriate  bridges  and  gateways. 

During  this  period  of  performance  the  connections  were  finalized  for  file  transfer  and  multiple- 
system  access  through  network  integration  and  remote  electronic  communications.  The 
interconnection  of  the  Performance  Network  to  the  other  networks  is  functional  but  relatively 
inflexible.  The  ARCnet  Hub  is  connected  to  an  ARCnet  interface  card  in  the  Banyan  DTS  file 
server.  With  this  arrangement  PCs  attached  to  the  Hub  can  be  booted  onto  the  Banyan  network. 
However  they  cannot  be  part  of  the  Banyan  Network  and  the  Performance  Network  at  the  same 
time.  That  is,  files  cannot  be  transferr^  from  a  Nestar  file  server  directly  to  the  Banyan  file 
server.  However,  a  PC  connected  to  the  ARCnet  Hub  can  download  files  from  the  VAXes  or 
the  Banyan  or  Nestar  file  servers  to  its  local  disk  then  reboot  on  the  appropriate  network  and 
transfer  the  file  to  the  network  file  server. 

A  commercial  phone  line  was  installed  in  the  Performance  Network  area.  One  PC  on  the 
ARCnet  hub  was  configured  with  a  modem  and  is  set  up  as  a  communications  server.  In  this 
mode  a  user  can  call  into  the  PC  from  a  remote  location,  go  through  the  Hub  to  the  Banyan 
network  and  on  to  the  VAX  network  if  desired. 

Communication  with  other  performance  laboratories  using  similar  Nestar  networks  has  been 
accomplished  with  the  Cc:Mail  software  from  PCC  Systems.  An  upgraded  version  of  that 
software  package  was  installed  on  the  Planstar  File  Server  during  this  period  of  performance. 
It  allows  both  incoming  and  outgoing  electronic  mail/file  transfer  capabilities  from  this  location. 

Storage  and  Security 

The  mass  storage  on  the  cluster  itself  is  currently  over  2  gigabytes  on  removable  and  fixed  disks. 
With  the  added  capabilities  of  the  MicroVAX  systems,  the  total  storage  in  the  AESOP  nears  4 
gigabytes.  A  1600/6250  bit  per  inch  tape  drive  provides  archival  storage  and  software  transfer 
for  the  cluster. 
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Other  peripherals  include  a  600  line  per  minute  LXY-22  line  printer  and  an  LN03  laser  printer 
as  cluster  printers.  All  systems  are  powered  through  a  centi^  distribution  unit  and  protected 
from  power  surges,  overheating,  and  fire  by  purity  and  monitoring  systems.  Maintenance  for 
both  hardware  and  software  is  covered  by  separately  funded  Air  Force  contracts. 
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AESOP  and  Local  Area  Networks 


Performance  Network 


Administrative  Network 


Branch 


Cluster  and  Simulation  Network 


HSO 


Banyon  to  DSRVB 


THE  AIRCREW  EVALUATION  SUSTAINED  OPERATIONS  (AESOP) 

FACILITY 


SRL  designed  and  oversaw  the  completion  of  a  3600  square  foot  research  facility  in 
the  high  bay  area  of  Building  1 70,  Brooks  AFB,  Tx.  The  AESOP  facility  houses  the 
generic  workstations,  computer  systems,  and  associated  hardware  and  is  designed 
to  specifically  meet  the  requirements  of  the  workstations  and  the  anticipated  research. 


Background 

in  the  spring  of  1986,  preliminary  plans  were  initiated  by  USAFSAM,  in  consultation 
with  the  Naval  Air  Test  Center,  to  design  a  research  laboratory  that  could  be  installed 
inside  the  high  bay  area  of  Building  170.  The  research  facility  had  to  be 
environmentally  controlled  and  reconfigurabie.  Both  the  AESOP  facility  and  the  C^GW 
hardware  and  simulators  are  designed  for  sustained  operations  research.  This  type 
of  investigation  requires  around-the-clock  operation  of  the  facility,  maintaining  a 
subject  pool  available  at  all  times,  special  considerations  for  hardware  maintenance 
and  data  backups,  and  complex  schedules  for  testing  and  training.  It  also  demands 
that  the  laboratory  accommodate  a  team  of  up  to  6  people,  with  total  control  over 
their  environment  and  isoiation  from  external  distractions. 

The  facility  needed  to  accommodate  extensive  cabling  between  computers, 
simulators,  data  collection  and  analysis  equipment,  and  scientific  workstations.  Data 
collection  required  electrically  noise-free  environments  and  isolated  circuits.  Data 
reduction  and  analysis  required  access  to  multiple  computer  systems  from  intelligent 
workstations. 


Design  Factors 

The  following  design  factors  were  included  to  meet  the  demands  presented  above: 

All  flooring  where  the  main  computers,  crewstations,  and  electronic  data- 
collecting  equipment  reside  were  raised  1 2"  above  ground  ievel  to  allow  ease 
of  cabling.  In  addition,  vertical  and  horizontal  wireways  were  integrated  into 
the  removable  wail  panels. 

All  electrical  circuits  to  provide  power  to  the  equipment  were  designed  to 
provide  the  best  possible  power  isolation,  shielding,  and  grounding  for  long¬ 
term  electrophysiologicai  recording  and  analysis.  All  electrical  circuits  are 
single  phase  to  a  given  lab  and  can  be  disabled  at  a  single  switch. 

Labs  are  equipped  with  incandescent  lighting  to  reduce  the  electromagnetic 
interference  in  experiments  requiring  precise  low-voltage  physiological 
recordings.  Fluorescent  lighting  was  restricted  to  highest  quality  fixtures. 
Areas  with  fluorescent  lighting  were  equipped  with  special  parabolic  wedge 


Scientific  workstations  were  strategically  placed  to  allow  access  to  data  for  off-line 
reduction  and  analysis  by  scientific  staff. 

The  structure  was  designed  to  be  used  for  sustained  operations.  All  areas  are  accessible 
without  having  to  exit  the  laboratory.  In  addition,  the  design  allows  restriction  of 
subjects  to  those  areas  designated  for  research.  The  two  floors  share  an  internal  stairway 
so  people  can  be  confined  to  the  facility.  Areas  on  the  second  floor  were  designed  to 
accommodate  eventual  use  as  housing/laboratory  areas.  Mission  debriefing  areas  and 
data  analysis  areas  were  designed  to  meet  the  unique  needs  of  the  Crew  Performance 
Function. 

In  order  to  achieve  full  environmental  control  for  both  computers  and  subjects  during 
sustained  operations  research,  the  research  laboratory  has  a  dedicated,  filtered  air 
conditioning  system. 

The  emphasis  on  performance  measures  requires  environments  as  free  from  distractions 
as  possible.  This  requires  that  every  effort  be  made  to  reduce  unwanted  sound  in  the 
research  areas.  Therefore  the  removable  panels  and  ceilings  were  designed  to  minimize 
noise  transmission  through  acoustically  insulated  walls. 

Sustained  operations  and  the  abundant  electronic  equipment,  along  with  local  fire  codes, 
required  a  fully  operational  fire  protection  system  that  operates  24  hours  a  day  in  an 
unattended  mode. 

The  expansion  area  will  provide  environmentally  controlled  space  for  large  screen 
displays  to  tie  into  the  crewstations  and  the  computers. 

All  of  these  design  features  were  incorporated  into  the  AESOP  facility,  since  no  other  existing 
structure  met  the  specifications  to  provide  the  proper  environment  for  this  type  of  research. 
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Performance  Capabilities 
and  Strategies 


GENERALITY 


Figure  1.  Performance  Measurement  Hierarchy. 


RESEARCH  AND  DEVELOPMENT  SOFTWARE  DOCUMENTATION 
Overview 

The  primary  activity  this  reporting  period  has  been  the  development  of  crewstation  and  simula¬ 
tion  software,  data  reduction  procedures,  and  data  formatting  for  statistical  analysis.  The  data¬ 
base  was  acquired  from  Airborne  Warning  and  Control  Systems  (AW ACS)  Weapons  Directors 
(WDs)  participating  in  simulation  exercises  on  the  Command,  Control  and  Communication  Gene¬ 
ric  Workstation  (C’GW). 

AWACS  Simulation  Software 

Euozfisfi 

The  models  comprise  the  basic  software  used  to  simulate  the  AWACS  crewstation. 
The  model  descriptions  in  this  section  will  be  most  useful  to  the  systems  programmer. 
The  systems  programmer  will  find  how  to  communicate  between  different  parts  of  the 
simulation  software  in  a  top-level  manner. 

Crewstation  Software 

The  AWACS  crewstation  software  consists  of  several  programs  arranged  in  a  hierar¬ 
chical  structure.  The  foundation  is  provided  by  the  resident  operating  system,  either 
VAX  VMS  or  IRIS  UNIX.  This  level  supports  all  of  the  other  levels.  It  provides  basic 
computing  and  file  manipulation  capabilities.  Without  the  operating  system  level,  the 
other  levels  could  not  function. 

Above  the  operating  system  level,  but  below  the  level  containing  the  modeling  pro¬ 
grams,  is  a  pseudo-level  called  the  hardware  dependent  layer.  This  layer  supports  the 
display  programs,  and  gives  them  independence  from  hardware  concerns.  The  hard¬ 
ware  dependent  layer  allows  the  display  programs  to  function  regardless  of  whether 
they  are  using  a  Curses-supported  terminal  or  a  Silicon  Graphics  workstation.  (Curses 
is  a  terminal  screen  handling  and  optimization  package.) 

The  top  level  contains  the  programs  that  simulate  the  AWACS  crewstation.  Each  of 
these  programs  is  generally  called  a  model.  Each  of  these  models  performs  a  small, 
relatively  independent  portion  of  the  total  task.  For  example,  the  Display  model  is  res¬ 
ponsible  for  updating  the  screen;  the  Target  model  is  responsible  for  updating  the 
position  of  aircraft;  and  the  Logger  model  is  responsible  for  recording  simulation 
events  for  later  study.  Each  model  performs  a  different  role  in  the  task  of  simulating 
the  AWACS  crewstation.  Figure  1  depicts  the  structure  of  the  AWACS  crewstation 
software. 

Model  Communication 


The  simulation  models  communicate  with  one  another.  The  information  exchanged 


is  divided  into  two  categories:  status  information  and  event  information.  Status 
information  includes  data  such  as  the  current  position  of  all  aircra^.  Event 
information  includes  data  such  as  "a  new  aircraft  should  be  added  to  the  simulation." 
Status  information  is  stored  in  the  Simulation  Data  Base.  The  Simulation  Data  Base 
is  discussed  in  the  section.  Data  Base  Methodology.  It  is  accessible  to  all  models. 
Figure  2  depicts  the  major  data  flows  among  the  models. 

During  the  initialization  of  the  simulation  software,  the  Broadcast  and  Target  models 
each  create  a  network  mailbox  on  the  VAX.  While  all  the  IRIS  target  models  (named 
STARGET)  connect  to  the  VAX  Target  model,  all  other  models  connect  to  Broadcast's 
mailbox.  Host  and  servers--VAX  and  IRIS  systems--must  complete  their  handshaking 
using  network  status  messages  before  any  communications  can  occur. 

Both  operating  systems,  VMS  and  UNIX,  provide  a  mailbox  system.  The  simulation 
communication  system  is  built  on  top  of  this.  Event  information  is  passed  from  one 
model  to  another  in  units  called  packets.  Packets  are  placed  by  the  sending  model 
into  the  mailbox  of  the  receiving  model.  The  receiving  model  retrieves  the  packet 
from  its  mailbox  to  obtain  the  event  Information.  Figure  3  depicts  AWACS  packet 
communication. 

When  a  model  receives  a  packet,  it  has  no  idea  of  "who”  sent  the  packet  (unless  that 
information  is  included  in  the  packet).  The  mailbox  mechanism  does  ofil  place  restric¬ 
tions  on  the  packet  sender.  Any  given  type  of  packet  will,  however,  usually  originate 
at  only  one  model.  Certain  types  of  packets  are  sent  by  the  Target  model  software. 
Other  types  of  packets  are  sent  by  the  Display  model  software.  Still  other  types  of 
packets  are  never  sent  by  the  software  directly;  some  packets  are  only  sent  when 
explicitly  requested  by  the  experimenter  or  scenario  designer. 

There  is  a  significant  difference  in  the  method  of  interrupt  notification  and  processing 
used  by  both  systems  for  normal  message  traffic.  The  VAX  interrupts  are  one  for 
one,  whereas  the  IRIS  can  develop  a  many  for  one,  i.e.,  one  interrupt  can  signify  that 
one  or  more  mailbox  messages  are  available. 


Model  Descriptions 

When  reading  the  model  descriptions  below,  refer  to  Figure  4,  AWACS  Simulation 
Overview,  for  a  graphic  depiction  of  simulation  software  and  hardware. 


Broadcast  Model 

The  Broadcast  model  serves  as  the  nrimary  communications  server  between  all 
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models.  All  models,  except  the  STarget  models,  create  a  communications  connection 
between  themselves  and  the  Broadcast  model.  The  major  function  of  the  Broadcast 
model  Is  to  route  packets  to  the  correct  destination  models. 

Any  model  may  send  packets  to  the  Broadcast  model.  These  packets  will  either  be 
directed  to  the  Broadcast  model,  or  will  contain  routing  information  so  that  the 
Broadcast  model  can,  in  turn,  send  the  packets  to  the  appropriate  model(s).  The 
Broadcast  model  can  route  packets  to:  a  single  model  by  name  (i.e..  Logger),  a  group 
of  models  by  name  (i.e.,  all  Display  models),  or  to  Display  models  having  a  specified 
assignment  type  (i.e.,  all  Simulation  Pilot  consoles). 


Display  Model 

The  Display  model  provides  two  types  of  functions:  Switch  Actions  and  Display 
Operations.  Switch  Actions  result  from  the  user  taking  some  action,  such  as  pressing 
a  switch  or  a  key,  and  imitate  the  true  AWACS  crewstation  capabilities.  Switch 
Actions  result  from  operator  inputs  entered  via  the  Action  Select  Buttons,  the  Feature 
Select  Switches,  the  Alarm/Alerts  Panel,  and  the  keyboard.  Also  included  in  this  cate¬ 
gory  are  actions  that  occur  periodically.  These  functions  are  driven  by  the  AWACS 
clock,  which  "ticks”  in  major  cycles  of  10  seconds,  and  minor  cycles  of  about  2.3 
seconds. 

Display  Operations  drive  the  display  hardware.  These  functions  are  completely  Igno¬ 
rant  of  the  AWACS  crewstation.  They  control  painting  lines,  circles,  and  text  in 
appropriate  colors.  These  functions  form  the  interface  to  the  display  hardware.  There 
are  currently  two  versions  of  the  Display  model.  One  version  contains  Display  Opera¬ 
tions  functions  that  drive  a  Silicon  Graphics  IRIS  4D/50.  The  other  version  contains 
functions  that  drive  a  standard  computer  terminal.  Both  versions  share  the  same 
Switch  Action  functions  (at  the  software  source  level). 

TargeLMddei 

The  Target  model  maintains  the  Simulation  Data  Base.  The  Simulation  Data  Base 
stores 


the  position  and  speed  of  aircraft, 
the  location  of  sensor  data,  and 

information  about  airbases,  lines,  circles,  and  special  points. 

The  Target  model  updates  the  position  of  moving  objects  within  the  Simulation  Data 
Base  on  receipt  of  timing  signals  from  the  Scenario  Manager  model.  The  Simulation 
Data  Case  contains  all  status  information  that  is  shared  between  models.  The  Target 
model  is  responsible  for  maintaining  all  of  the  Simulation  Data  Base.  Information  that 
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must  be  available  to  multiple  models  is  placed  in  the  Simulation  Data  Base. 


Switch  Model 

The  Switch  model  Is  used  to  monitor  the  crewstation  switches.  It  Is  responsible  for 
responding  to  Display  model  requests  for  crewstation  lights  to  go  on  or  off.  Each 
Switch  model  is  associated  with  a  Display  model.  Upon  recognizing  a  switch  press, 
the  Switch  model  sends  a  packet  to  the  Display  model  describing  the  switch  change. 
The  Display  model  may  respond  by  sending  a  packet  to  the  Switch  model  containing 
instructions  on  modifying  the  state  of  the  crewstation  lights. 


Speech  Model 

The  Speech  model  is  used  to  produce  computer-generated  speech.  The  Speech  model 
controls  the  two  VOTAN  VTR-6050  speech  devices  used  by  the  Simulation.  The 
Speech  model  initializes  the  VOTANs,  and  responds  to  requests  to 

reset  either  VOTAN, 

load  a  voice  file  on  either  VOTAN,  and 

to  "speak"  a  voice  message. 

Each  of  the  VOTANs  is  connected  to  one  of  the  ADS  frequencies,  so  messages  played 
on  each  VOTAN  will  be  heard  on  the  appropriate  frequency. 


Logaer  Model 

The  Logger  model  records  events  detected  during  a  simulation.  It  stores  the  log  of 
events  as  a  standard  text  file,  with  each  event  occupying  one  line  of  the  file.  Each 
event  is  stored  immediately  in  the  log  file;  no  interpretation  is  attempted  during  the 
simulation.  The  log  file  exists  so  any  event  interpretation  desired  can  be  performed 
after  the  simulation  completes.  The  record  of  each  event  includes 

-  the  time  at  which  the  event  occurred, 

-  the  model  recognizing  the  event, 

-  the  type  of  event,  and 

-  the  data  associated  with  the  event. 

The  event  time  is  recorded  ir  Multiport  Clock  Time,  or  in  IRIS  TIMERO  Time.  Multiport 
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Clock  Time  is  very  precise  with  ticks  of  approximately  32  microseconds.  IRIS  TIMERO 
Time  Is  less  precise  with  ticks  of  approximately  17  milliseconds.  It  is  important  to 
note  that  while  all  event  times  are  stored  with  very  high  precision,  not  all  of  them  are 
measured  with  such  accuracy.  Switch  presses  and  key  presses  are  measured  with 
the  highest  accuracy  possible.  Less  care  is  taken  when  measuring  other  events. 

The  Logger  model  accepts  only  one  kind  of  packet.  This  packet  contains  a  description 
of  an  event.  The  event  description  consists  of  the  four  items  listed  above,  time, 
source  model,  type,  and  data.  Any  of  these  items  may  be  omitted  from  a  packet.  If 
time  is  omitted  from  the  packet,  the  Logger  model  records  the  current  Multiport  Clock 
Time  as  the  event  time.  Since  there  is  a  time  lag  present  between  the  time  the  event 
is  recognized,  and  the  time  that  the  Logger  model  receives  the  packet,  some  error  is 
introduced  by  this  mechanism.  Each  of  the  other  items  receives  a  null  value  if 
omitted. 


Scenario  Manager  Model 

The  Scenario  Manager  model  is  responsible  for 

updating  the  simulation  time  (an  integer  counting  in  cycles)  and  the 
scenario  date/time  (a  digital  clock  format);  and 

sending  the  scenario  packets  to  the  other  models  at  the  appropriate  times. 

The  Scenario  Manager  model  reads  the  Scenario  File  into  memory  at  the  very  begin¬ 
ning  of  the  simulation  run.  It  increments  the  simulation  time  and  generates  the 
scenario  date/time  based  on  the  simulation  time  and  the  Scenario  Base  Date.  Both  the 
simulation  time  and  the  scenario  date/time  are  maintained  in  the  Simulation  Data  Base. 
A  simulation  time  is  attached  to  each  scenario  packet.  As  each  scenario  packet 
becomes  due,  the  Scenario  Manager  model  sends  it  to  the  appropriate  model. 

The  first  entry  in  every  scenario  file  must  be  a  line  specifying  the  Scenario  Base  Date. 
The  line  must  be  in  the  form: 

<day>  <hour>  <  minute  > 

Each  of  the  fields,  day,  hour,  and  minute  are  Integers.  <day>  is  the  Julian  Date  of 
the  Scenario  Base  Date.  <hour>  is  the  hour  of  the  day  in  military  time  (  00  hours  - 
24  hours).  < minute >  is  the  minute  of  the  hour. 

To  specify  that  the  scenario  begins  at  1 :29  p.m.  on  March  27  of  a  leap  year,  the  first 
line  of  the  scenario  file  would  be: 

087  13  29 
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During  a  scenario  run,  the  current  date/time  is  computed  by  adding  the  current  simula¬ 
tion  time  to  the  Scenario  Base  Date.  This  is  an  unusual  operation  since  the  Scenario 
Base  Date  is  measured  in  minutes  and  the  simulation  time  is  measured  in  cycles 
(tenths  of  seconds).  If  the  Scenario  Base  Date  is  taken  from  the  example  above,  and 
the  simulation  time  is  655,  then  the  scenario  date/time  will  be  087  13:30:05.5 
(65.5  seconds  after  087  1 3:29:00.0).  The  default  Scenario  Base  Date  is  the  date  and 
time  at  which  the  Simulation  Data  Base  is.  initialized. 
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Figure  1.  Structure  of  the  AWACS  Crewstation  Software. 


Figure  2.  AWACS  Major  Data  Flows. 


Figure  3.  AWACS  Packet  Communication. 


Figure  4.  AWACS  Simulation  System. 


PERFORMANCE  MEASUREMENT  METHODOLOGY 

Of  primary  importance  to  this  study  was  the  assessment  of  drug  effects  on 
team  decision  making  in  performing  complex  tasks.  Although  there  are  several  models 
for  evaluating  teams,  most  require  inputs  from  trained  observers  making  subjective 
ratings.  Reliable  detection  of  subtle  medications  and  fatigue  effects  requires 
objective,  repeatable  measures.  After  a  review  of  team  performance  literature,  Eddy 
( 1 989)  concluded  that  no  one  has  systematically  developed  and  empirically  tested  a 
comprehensive  theory  of  team  performance.  As  a  result  we  developed  a  hierarchical 
performance  assessment  system  to  provide  structure  for  understanding  performance 
in  WD  tasks.  This  system  provides  an  implicit  underlying  structure  that  weights  the 
significance  of  each  measure  and  relates  it  to  the  others.  Each  level  of  the  hierarchy 
contains  groups  of  measures  that  jointly  determine  the  measures  available  at  the  next 
level  higher  in  the  framework.  This  system  includes  4  interrelated  levels  of  metrics 
(see  Figure  1).  From  the  top  down  the  levels  are: 

•  Mission  Effectiveness, 

•  System/Team  Performance, 

•  Individual  Performance,  and 

•  Performance  Capability. 

Each  level  of  the  Performance  Measurement  Hierarchy  was  developed  in 
conjunction  with  operationally  experienced  SMEs  in  AW  ACS  C*  tasks.  The  mission 
effectiveness  level  is  assessed  exclusively  by  ot/fcome  measures,  i.e.,  measures  of  the 
team's  results.  The  system/team  performance  level  is  assessed  by  several  types  of 
mufti-dimensional  measures  combined  to  quantify  changes  in  situational  awareness, 
cooperation,  cohesiveness,  adaptation,  and  distribution  of  work.  The  individuaf 
performance  measures  consist  mainly  of  process  measures.  Process  measures  are 
measures  of  activities  used  to  accomplish  the  mission  and  produce  the  final  results. 
They  include  task  completion  times  and  response  variability,  and  information 
processing  rates  as  they  relate  to  unique  task  assignment.  Performance  capabiiity  is 
measured  by  skill  assessment  batteries  administered  separately  from  the  scenarios. 


Mission  Effectiveness  measures  are  derived  directly  from  the  specific  objectives 
of  the  mission  assigned  to  the  system.  For  the  C’  AW  ACS  system  the  objectives 
include  the  following: 

1 )  protection  of  a  specific  sector  of  air  and  ground  space  from  infiltration  by 
enemy  aircraft  (protection  of  assets), 

2)  minimization  of  resource  expenditure  (fuel,  weapons)  in  protection  of 
assets,  and 

3)  maximization  of  resource  survivability  (interceptor  aircraft  as  well  as  self). 


Measures  that  flow  from  these  high-level  objectives  and  that  assess  perfor¬ 
mance  in  terms  of  Mission  Effectiveness  include,  among  others,  the  number  of  enemy 
Infiltrations,  fuel  and  weapons  expended,  and  the  ratio  of  systems  returning  to  sys¬ 
tems  deployed. 


Svstem/Team  Performance  Measures 

The  second  level  of  the  hierarchy,  System/Team  Performance,  contains  groups 
of  measures  reflecting  factors  that  ImmedIcJtely  affect  Mission  Effectiveness.  These 
include  the  threat  environment  (composition  and  performance  of  enemy  forces),  the 
physical  environment  (weather,  etc.),  and  the  performance  of  the  C®  system  itself. 
Since  the  emphasis  of  the  simulations  was  to  measure  the  factors  under  at  least  par¬ 
tial  control  of  the  human  operator,  it  was  the  latter  group  of  determinants  that  was 
of  interest. 

Such  measures  of  System/Team  Performance  reflect  the  degree  to  which  the 
combined  man-machine  system  has  accomplished  those  tasks  required  to  meet 
mission  objectives.  These  metrics  do  Ufll  reflect  the  individual  contributions  of 
different  human  behaviors  or  various  hardware  and  software  component  perfor¬ 
mances.  instead,  they  are  more  global  indices  of  the  degree  to  which  the  total  system 
successfully  accomplished  the  tasks  essential  to  mission  success. 

In  order  to  derive  such  measures,  it  was  necessary  to  obtain  a  detailed  descrip¬ 
tion  of  the  specific  methods  by  which  the  system  accomplishes  its  mission.  For 
example,  the  weapons  director/workstation  system  is  required  to  meet  its  mission 
objectives  by  accomplishing  a  weapons  control  function  aimed  at  directing  interceptor 
aircraft  to  defeat  threat  aircraft.  This  weapons  controller  task  was  broken  down  Into 
a  number  of  essential  subtasks  such  as  pairing  of  interceptors  with  targets,  providing 
target  data  to  interceptors,  and  maintaining  target  correlation,  among  others.  Perfor¬ 
mance  measures  of  these  system  tasks  include  the  proportion  of  time  targets  are  un- 
correiated  and  the  accuracy  and  speed  of  data  transfer  to  interceptors,  among  others. 

Individual  Performance  Measures 

The  third  level  of  the  hierarchy.  Individual  Performance,  contains  process 
measures  that  assess  the  individual  contributions  of  hardware/software  and  human 
components  to  overall  system  performance.  Measures  of  the  Individual  Performance 
level  of  the  hierarchy  are  designed  to  reflect  the  quality  of  the  individual  behaviors 
required  of  the  WD  expressed  primarily  in  terms  of  latencies,  errors,  and  rate  of  cor¬ 
rect  responses.  These  metrics  are  derived  by  examining  the  system  functions  required 
to  meet  mission  objectives  to  identify  the  specific  contributions  of  the  operator.  For 
example,  the  system  performance  requirement  to  pair  targets  with  interceptors 
requires  the  WD  to  identify  a  target's  location  on  the  workstation  display,  and  commu¬ 
nicate  this  information  to  an  interceptor  aircraft  via  radio.  The  quality  of  the  opera¬ 
tor's  performance  in  achieving  this  objective  can  be  measured  by  evaluating  the  time 
needed  to  complete  the  full  sequence  of  required  behaviors  and  by  assessing  the 
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accuracy  of  each  manual  and  verbal  response. 

In  deriving  the  Individual  Performance  measures  it  is  crucial  to  ensure  that  the 
aspect  of  performance  assessed  is  a  true  contributor  to  system  performance.  For 
example,  assessing  response  time  on  a  task  component  ngt  time-critical  could  easily 
lead  to  erroneous  conclusions  about  the  operator's  performance. 

Performance  Capability  Measures 

The  final  level  of  the  hierarchy.  Performance  Capability,  contains  measures  that 
assess  factors  directly  affecting  the  individua]  performance  capacities  of  primary  sys¬ 
tem  components.  For  hardware,  these  measures  might  include  data  transfer  rates, 
component  reliabilities,  etc.  For  the  human  operator,  measures  of  Performance  Capa¬ 
bility  are  composed  of  a  large  group  of  potential  human  state  and  ability  metrics  that 
combine  to  determine  overt  performance.  These  metrics  include  Indices  of  workload 
or  reserve  processing  capacity,  fatigue,  mood,  arousal  level,  experience  level,  and  indi¬ 
vidual  perceptual,  cognitive  and  motor  abilities  that  make  up  the  total  productivity  of 
the  operator. 


Hierarchical  Relationships 

The  multi-level  classification  of  performance  measures  has  the  advantage  of 
placing  metrics  into  logical  subordinate  and  superordinate  groups  that  indicate  the  pre¬ 
dictive  relationships  among  them.  In  addition,  measures  at  each  of  the  levels  differ 
in  their  sensitivity,  generaiizability  and  practical  interpretability.  Examining  the  hier¬ 
archy,  it  is  obvious  that  the  data  provided  by  the  highest  level  of  measurement  Is 
easily  Interpreted  while  lower  levels  offer  information  increasingly  remote  from  the  ulti¬ 
mate  criterion  of  mission  success  or  failure.  However,  this  disadvantage  is  countered 
by  the  fact  that  measures  at  lower  levels  of  the  framework  are  both  more  sensitive 
and  general  than  those  at  higher  strata.  For  example,  while  kill  ratios  are  direct 
indices  of  Mission  Effectiveness,  these  measures  are  influenced  by  a  host  of  individual 
factors  that  make  them  insensitive  to  small  but  significant  variations  in  such  things 
as  operator  decision  time.  Furthermore,  Mission  Effectiveness  measures  are  highly 
specific  to  the  individual  characteristics  of  the  test  scenario.  Hence,  an  effectiveness 
metric  obtained  under  one  set  of  conditions  may  give  little  indication  of  the  system's 
performance  in  a  different  situation.  Conversely,  a  measure  of  operator  reserve  capa¬ 
city,  such  as  a  response  time  on  an  embedded  secondary  task,  is  difficult  to  relate 
directly  to  a  criterion  such  as  survivability.  At  the  same  time  however,  such  a 
measure  is  generalizable  across  a  wide  range  of  simulation  scenarios,  and  will  be  ex¬ 
tremely  sensitive  to  variations  in  operator  capability. 

These  features  of  the  different  levels  of  performance  measurement  make  it 
extremely  important  to  identify  the  specific  assessment  goals  of  a  system  simulation 
in  order  to  ensure  appropriate  data  are  collected.  Since  a  primary  goal  of  the  simula¬ 
tions  was  to  explore  the  impact  of  operator  variables  on  system  and  mission  perfor¬ 
mance,  it  was  necessary  to  collect  detailed  measures  of  Mission  Effectiveness  and 
System  Performance  in  order  to  identify  operationally  significant  effects  of  the 
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medications  and  stressor  variables.  However,  because  of  the  predicted  limited  sensi¬ 
tivity  and  generality  of  these  measures.  It  was  also  necessary  to  obtain  measures  from 
the  lowest  levels  of  the  performance  hierarchy.  Such  Individual  Performance  and  Per¬ 
formance  Capability  metrics  extend  the  utility  of  necessarily  constrained  research 
studies  and  permit  generalization  to  a  wide  range  of  systems  and  mission  scenarios. 

Correlating  Measures 

In  attempting  to  measure  complex  decision-making  performance,  correlations 
with  other  simpler  performance  measures  should  be  explored.  These  simpler 
measures  may  be  predictive  of  the  complex  decision-making  performance.  If  the  sim¬ 
pler  measures  are  found  to  be  predictive,  they  may  be  useful  in  selecting  future  WDs. 


The  study  used  several  classes  of  measures:  cognitive  and  psychomotor 
performance  measures,  standardized  complex  task  measures,  sleep  survey,  mood 
scale,  fatigue  scale,  subjective  workload  scale,  biographical  sketch,  WD  experience, 
and  personality  measures. 


